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We provide a short and elementary proof for the recently proved result by G. da Prato and 
H. Frankowska that - under minimal assumptions - a closed set is invariant with respect 
to a stochastic control system if and only if it is invariant with respect to the (associated) 
l— ~~ '■ deterministic control system. 
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[> 1 Introduction 

o ; 
r- , 

We deal in this note with invariance of controlled stochastic differential systems. We consider 
| a non-empty, closed subset K C IR™ and ask for characterizations of invariance of K with 
respect to a controlled stochastic differential system 

m dX t = b{X t ,ut)dt + a{Xt,Ut)dW u t>0, 

[) X = xeR n , 

driven by a d-dimensional Brownian motion W. 

Invariance of K here means that P[X^' U E K\ = 1, for all x G K, all times t > and all 
admissible control processes u. 

There exist already a lot of literature concerning invariance as well as the connected no- 
tion of viability; characterizations of both have been expressed through stochastic tangent 
cones (PP, pj]), viscosity solutions of second-order partial differential equations (e.g. see 
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[3] j [I] j [I] > [S] , P3] ) or other approaches (e.g. see [8],[14J). 

A natural approach of the notion of invariance is to look at the associated controlled ordinary 
differential system: 

(] x'{t) = ~b(x{t),u{t)) + a(x(t),u(t))v(t), t>0, 

1 ] x(0) = x, 

where b(x,u) denotes the Stratonovich drift b(x,u) = b(x,u) — \ Y,i=i(D x cr l (x, u), a l (x, u)) 
and v G Lj oc ([0, oo),R n ). For the case without control it is well known that invariance with 
respect to is equivalent to invariance with respect to the ordinary differential system 
(see |10|.[16]. [2]). Recently G. da Prato and H. Frankowska [9] proved the result on the 
equivalence for controlled deterministic and stochastic systems under minimal assumptions 
on the involved parameters. Our aim here is to provide a new, short and very elementary 
proof of this intuitive equivalence result. 

The intuition behind our main result stems from the local asymptotics of the stochastic 
systems, which correspond precisely to those of the deterministic system. Reading this 
insight, which is well-known in numerical analysis for the given stochastic differential system, 
in the correct way, leads us to the proof. A central step in our investigation is to show 
that stochastic as well as deterministic invariance is equivalent to invariance with respect 
to constant controls. This permits us to pass from deterministic invariance to stochastic 
invariance by the classical Wong-Zakai-approach to martingale problems (which can be seen 
as a sort of Euler-Mayurama-scheme, too). Concerning the other direction of the proof, the 
necessary conditions on the parameters follow naturally from a stochastic Taylor expansion. 
As a crucial tool we apply optimization theory, since both invariance problems can be 
associated with problems of minimal distance to K. Hence we can also assert an equivalence 
between first and second order Hamilton- Jacobi-Bellman systems. 



2 Main Theorem 

Let U be some compact metric space and, for d, n > 1, let b be a bounded and continuous 
map from R" x U to R" , Lipschitz in x G R n uniformly in u G U, and a a continuous map 
fromR™ x U toR nxd , differentiable with respect to x, such that a and D x a are bounded and 
Lipschitz, both uniformly in u. 

Let W be a d- dimensional Brownian motion on some probability space (Q, P) and (jF t , t > 
0) the filtration generated by W, satisfying the usual assumptions. We denote by U the set 
of all ^/-valued processes (u t ) that are progressively measurable w.r.t. {T^t > 0). 
For (u t ) G U, we consider the controlled stochastic differential system : 

dX t = b{X u u t )dt + a{X t) u t )dW t , t>0, 
[) I = xGt n 

It is well known that under the above assumptions on b and a, the system ([3]) has a unique 
strong solution, which we denote by X x ' u . 

We associate to this system the usual second order operator: for ip G C 2 (R n ,R), x GR™ and 
u G U, 

£x,u¥ = (b(x, u),<p(x)) + -tr(D 2 ip(x)a(x, u)a*(x, u)). 
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We denote by b the Stratonovich drift 

1 d 

b(x,u) = b(x,u) - - ^2(D x a\x,u),a\x,u)), 
1 i=i 

where a l (x,u) is the i-th column of the matrix a(x,u). 

Furthermore we consider a non empty closed set K C BL n . The notion of invariance of K 
with respect to ([3]) is defined as follows: 

Definition 2.1 We say that K is invariant with respect to ([3]) if, for all x G K , u G IA, 

andt> 0, P[X?' U eK] = l. 

We also shall introduce the deterministic system : 

U) x'(t) = b(x(t),u(t)) + a(x(t),u(t))v(t), t>0 

[ } x(0) = x, 

driven by the deterministic control process v (t) G B := Lj oc ([0,+oo),R d ) and u(t) G A := 
L°°([0, oo),U). For given x G R n , u G A and dgB, the solution of flU will be denoted by 
x x ' u ' v . The associated first order operator is, for ip G C 1 ^™), 

£'x,u¥ = (b(x,u),D(p(x)). 

Definition 2.2 We say that a closed set K is invariant w.r.t. if, for every x G K, 
u G A and v G B, x x ' u ' v (t) G K for every t > 0. 

For ip : 1R" — >1R, we denote by Argmax^y? the set of x G K such that ip attains a maxi- 
mum at x in K. 

Our main result is the following theorem. 

Theorem 2.1 The following assertions are equivalent : 

a) K is invariant with respect to (TJj)/ 

b) For all (p G C 2 and x G Argmax K ip , it holds that 



(5) 



(a i (x,u),Dtp(x)) = 0, Vz G {!,..., 4, VmG *7; 



cj For all (p G C 2 and x G Argmax K ip , it holds that 
' sup„ eC7 £' <£>(» < 0, 



(6) 



(a J '(£,4,£V(x)) = 0,VzG {1,...,4,VmG E/, 

t/ie matrix A^ x = (ay) itra^/i a,j = (a l (x,u), D x (a :i (-,u), Dip(-))(x)) 
is symmetric and semidefinite negative; 



d) K is invariant with respect to (fj]j; 

e^) For a// G C 2 and x G Argmax K ip , it holds that 

(7) sup {£^0) + (o(x, Dip(x))} < 0. 
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Remark 2.1 Applying the notations from differential geometry b u ip(x) := (b(x, u), Dip(x)) 
and a l u (p(x) := (a l (x,u), Dtp(x)) , condition can be rewritten as follows: 
For all u £ U, it holds that 

b u <p(x) < 0, 

oi<p(x) = o,yie{i,...,d}, 

(o"*cr^(x))y is symmetric semidefinite negative. 
The following Lemma is crucial in the proof of the Theorem. 

Lemma 2.1 Let (W t )t>o be a standard HL d -valued Brownian motion issued from and 
(Rt)t>o a rea l stochastic process satisfying lim t ^ -y- = m probability. 
Let (a,, 1 < % < d) £ R d , 1 < % < d) £ R d , (7^, (i, j) £ {1, . . . , d} 2 , i ^ j) £ R d2 ' d and 
5 £BL Suppose that, for all t > 0, P-a.s., 

(8) Y, a i w t+J2&( w t) 2 + E njfwlwl + st+RtKQ. 

i=l i=l l<i^j<d 

Then it holds that 

i) ctj = 0, for all i £ {1, . . . , d}; 

ii) the matrix A £R dxd defined by 

f A ij = T*i> /° r (*> i) e {1, . . . , ci} 2 , i 7^ J) 
\ Ajj = 2pi,ie {!,..., d}, 

is symmetric and semidefinite negative; 
m) 5 < 0. 

Proof : It is easy to see that 

2^ Pi — 1=- + l^7ij 7= h dvi + -7= -> 0,as t \ 0, 

i=i vt ^ vt vt 

while, 

d W i rj\ d 

vt>o, E«^ = E«^i- 

8=1 Vt j=l 

It follows that the left hand term of (jHJ) divided by v^, say L t , converges in distribution to 
£?=i «iWf- Now the assumption P[L t < 0] = 1 for all t > implies that P[Ei=i Oi^i < 
0] = 1, too. It follows that, necessarily ct\ — . . . — a.4 — 0. 

Using again the scaling property of Brownian motion, we have, for all £ {1, . . . , d} 2 

and for all t > 0, | J * W^dW^ = /q WJdl^'. By the same arguments as above, we can 
deduce from (jSJ) that, P-a.s., 

(9) E A(w?) a + E 7« + & < 0. 

i=l i/J- J0 
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Let us focus now on a fixed arbitrary couple of indexes with % 7^ j. After conditioning 
by a{Wl Wj, s > 0), we get from ©, P-a.s., 

(10) #(W*) a + &(W?) 2 + 7y /' WldWi + 7ji f WidWt + 5+J2(3 2 k <0. 
Introducing the Levy area L tj = W^dW^ — Jq W^dW^, we can write : 

If we substitute this in (flQj) . it follows that, P-a.s., 

i( 7y - in)E[L^\Wl = W(=0] + 6+Y: Pi < 0. 

But, even after conditioning by W[ = W{ = 0, the distribution of L 1 ^ is symmetric and of 
unbounded support. Consequently it holds that 7 j = 7^. 

Since (z, j) G {1, . . . , d} 2 , z 7^ j was chosen arbitrarily, (jHJ) becomes now, P-a.s., 

8=1 i<j 

or, equivalently, 

(11) ^<Wi,AWi) +£<(). 

Since the support of W% isffi^, ii) and iii) follow. o 

Proof of the Theorem. We consider the following two additional assertions, where u aU 
is identified with the deterministic constant control process u t = u,t > 0. Notice that X x,u 
is defined by ([[]) and x x,t,u > v by (TJ]). 

f) For all u G U, x G K and t > 0, P[Xf' v G K] = 1. 

g) For all it G Z7, 2 G K and any admissible control u G £>, the function ar"'"'"^) takes 
its values in K. 

The proof will be organized as follows : 




e c ►- b 



• d)=>- g) is trivial. 
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• g) =>- f): We fix some u G U and consider a scheme which converges in distribution 
to the solution of the stochastic differential equation ([1]) with constant, deterministic 
control u t = u. For the construction we apply the following limit theorem [T21 Theorem 
1, p. 698]: For all t > 0, we set £ t = W t+ i — W t . The process (£t) t>0 is strictly stationary 
and ergodic. We let rj^ 1 = \frn^ mtl t > 0, m > 1, and put 

y m = [ l v ™ds,t>0. 
Jo 

Notice that y m is a stochastic process with differentiable trajectories. Furthermore, the 
process Y m converges w-wise, uniformly on compacts to W, as m — > oo. Consequently, 
it converges also in distribution on pathspace. Theorem 1 from [12] tells now that the 
unique solution of 

dX™ = b{X r t n ,u)dt + a(X™,u) dY t rn , A m = x , 

converges in distribution on pathspace to X x ' u . The conditions as stated in [12J on a 
are slightly stronger than our assumptions, namely C 2 is required. However, the proof 
in [12] also holds for a satisfying our C 1,1 -assumptions. Certainly we cannot deduce 
by P21 Theorem 1] a rate of convergence for X m — > X 31 '", but we also do not need 
such a rate for our purposes. 

We know that, by assumption, with probability 1, X™ G K for all x G X, t > and 
n > 1, whence we obtain the result: Indeed, if dx{x) denotes the distance from sGl" 
to if, we have 

E[d K (X t )) = lim E[d K (X™)} = 0, for all t > 0. 

L n^oo 

• f)=> c) : Consider a constant control u t = u £ U and suppose that, for all rc G if and 
£ > 0, P[Xf' u e K] — 1. Let tp <E C 2 and x G Argmax^. Up to change ip outside of 
some open set including x, we can suppose that ip, \\D(p\\ and ||PV|| are bounded. We 
can apply the stochastic Taylor expansion formula ([13] or [5]): for alH > 0, P-a.s., 

+ <<^>(x) W\dWl + b u ^{x)t + R t , 

where R t satisfies ^ — > in probability as t \ 0. We apply here the operator- 
notations a l u (p(x) = (a(x, u), Dip(x)) and b u <p(x) = (b(x,u),D(p(x)). 
Since K is invariant for the constant control u and since x G Argmax^t/?, we have 
P-a.s., for all t > 0, c^(Xf' u ) < Thus, P-a.s., for any fixed t > 0, 

+ £ ¥i oi<rl<p(x) Jo* WidWi + b u( p(x)t + R t <0. 
Now we can apply Lemma 12.11 and get exactly the claim. 
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c)=>- b) becomes trivial as soon we write 

b u ip(x) + ~tr(D 2 (p(x)cr(x, u)a*(x, u)) = -A^ x + b u ip(x). 

b) =^> a) : The proof is adapted from the equivalent result about viability in [6J. It is 
easy to see that, if b) holds, then the map / : x t— > 1 — i K (x) is a supersolution of 

mpC XiU f(x) = 0. 
ueu 

We consider now a constant C > 1 and an uniformly continuous application g from 
IR™ to [0, 1] that satisfies g(x) = if and only if x E K. Since, for all x E E. n , 
g(x) < Cf(x), f is also a supersolution of the following Hamilton- Jacob i-Bellman 
equation 

(12) sup C x , u f(x) + g(x) - Cf(x) = 0. 

u£U 

But we know that the unique solution V with polynomial growth of ( fl2|) can be 
represented as 

V{x) = sup E[ / e- c 'g{X? u )ds]. 
ueu Jo 

By the comparison theorem, we then have 

V(x) < f(x),xeR n . 
For x G K, this implies that, for all u<EU, for all t > 0, P[X^ U E K] = 1. 
a) =^> f) is trivial. 

c) =^ e) is trivial. 

e)=^ d) could be deduced from b)=>- a) if v would take its values in a compact space 
and if b and a would be replaced by suitable functions. Let us clarify this point: We 
fix x E K, v E B and u E A. We wish to prove that 

x x ' u ' v {t) E K, for all t > 0. 

For any integer n > 0, we can define the control t i— > v n (t) := 7T n (v(t)), where n n 
denotes the projection onto B(Q,n). 

By standard estimates, the sequence x x,u,Vn converges to x x,u ' v uniformly on every 
compact intervals [0, T\. Obviously x x,u,Vn is solution to the following control system 



(13) x'{t) = b(x(t),u(t)) + a(x(t),u(t))v(t),u(t) E U, v(t) E B(0, n),x(0) = x, 



with (u,v) taking values in the compact set U x B(Q,n). Hence we can apply the 
already proved relation b) =^ a) to the control system ffl3|) with b(x, u) replaced by 
b(x, u) + a(x,u)v, o replaced by and the control u replaced by (u,v). In this case 
the relation (jSJ) reduces to 

sup {C' x u (p(x) + (a(x, u)v, Dip(x))} < 0. 

ueU, veB(0,n)) 

Consequently, we deduce from e) that x x,u,Vn (t) E K, for all t > 0. By passing to the 
limit with respect to n, we obtain that x x,u ' v (t) E K forall t > 0. Our claim is proved. 
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